Visualizing recommendations to support exploration, transparency and controllability

Research on recommender systems has traditionally focused on the development of algorithms to improve accuracy of recommendations. So far, little research has been done to enable user interaction with such systems as a basis to support exploration and control by end users. In this paper, we present our research on the use of information visualization techniques to interact with recommender systems. We investigated how information visualization can improve user understanding of the typically black-box rationale behind recommendations in order to increase their perceived relevance and meaning and to support exploration and user involvement in the recommendation process. Our study has been performed using TalkExplorer, an interactive visualization tool developed for attendees of academic conferences. The results of user studies performed at two conferences allowed us to obtain interesting insights to enhance user interfaces that integrate recommendation technology. More specifically, effectiveness and probability of item selection both increase when users are able to explore and interrelate multiple entities - i.e. items bookmarked by users, recommendations and tags. Copyright © 2013 ACM

Layered evaluation of multi-criteria collaborative filtering for scientific paper recommendation

Recommendation algorithms have been researched extensively to help people deal with abundance of information. In recent years, the incorporation of multiple relevance criteria has attracted increased interest. Such multi-criteria recommendation approaches are researched as a paradigm for building intelligent systems that can be tailored to multiple interest indicators of end-users – such as combinations of implicit and explicit interest indicators in the form of ratings or ratings on multiple relevance dimensions. Nevertheless, evaluation of these recommendation techniques in the context of real-life applications still remains rather limited. Previous studies dealing with the evaluation of recommender systems have outlined that the performance of such algorithms is often dependent on the dataset – and indicate the importance of carrying out careful testing and parameterization. Especially when looking at large scale datasets, it becomes very difficult to deploy evaluation methods that may help in assessing the effect that different system components have to the overall design. In this paper, we study how layered evaluation can be applied for the case of a multi-criteria recommendation service that we plan to deploy for paper recommendation using the Mendeley dataset. The paper introduces layered evaluation and suggests two experiments that may help assess the components of the envisaged system separately. Keywords: Recommender systems; Multi-Criteria Decision Making (MCDM); Evaluatio

IntersectionExpIorer: The flexibility of multiple perspectives

Recommender systems are currently an ubiquitous presence on the web, helping us find relevant items in the ever-growing plethora of information available. However, there is not a one-size fits-all for recommender systems, and flexibility and control are crucial for enabling the possibility of adapting the recommender system to different user preferences. In this paper, we present the results of a study designed to assess user interaction with IntersectionExplorer (IEx), a multi-perspective tool for exploring conference paper recommendations. The study was conducted at the Digital Humanities 2016 Conference, an event with a rather large, heterogeneous, and not technology-oriented audience. The results obtained indicate that the IEx multi-perspective approach lends enough flexibility to accommodate different user preferences. When contrasting these results with a previous study conducted at a conference with a highly technological audience, it becomes apparent that the flexibility of IEx is key to empower users with different profiles to customize their approach to finding relevant recommendations

The effect of different set-based visualizations on user exploration of recommendations

When recommendations fail, trust in a recommender system often decreases, particularly when the system acts like a "black box". To deal with this issue, it is important to support exploration of recommendations by explicitly exposing relationships that can provide explanations. As an example, a graph-based visualization can help to explain collaborative filtering results by representing relationships among items and users. In our work, we focus on the use of visualization techniques to support exploration of multiple relevance prospects - such as relationships between different recommendation methods, socially connected users and tags. More specifically, we researched how users explore relationships between such multiple relevance prospects with two set-based visualization techniques: a clustermap and a Venn diagram. A comparative analysis of user studies with these two approaches indicates that, although effectiveness of recommendations increases with the use of a clustermap, the approach is too complex for a non-technical audience. A Venn diagram representation is more intuitive and users are more likely to explore relationships that help them find relevant items

<i>“We’re Seeking Relevance”</i>: Qualitative Perspectives on the Impact of Learning Analytics on Teaching and Learning

Whilst a significant body of learning analytics research tends to focus on impact from the perspective of usability or improved learning outcomes, this paper proposes an approach based on Affordance Theory to describe awareness and intention as a bridge between usability and impact. 10 educators at 3 European institutions participated in detailed interviews on the affordances they perceive in using learning analytics to support practice in education. Evidence illuminates connections between an educator’s epistemic beliefs about learning and the purpose of education, their perception of threats or resources in delivering a successful learning experience, and the types of data they would consider as evidence in recognising or regulating learning. This evidence can support the learning analytics community in considering the proximity to the student, the role of the educator, and their personal belief structure in developing robust analytics tools that educators may be more likely to use

Scalable exploration of relevance prospects to support decision making

Recent efforts in recommender systems research focus increasingly on human factors that affect acceptance of recommendations, such as user satisfaction, trust, transparency, and user control. In this paper, we present a scalable visualisation to interleave the output of several recommender engines with human-generated data, such as user bookmarks and tags. Such a visualisation enables users to explore which recommendations have been bookmarked by like-minded members of the community or marked with a specific relevant tag. Results of a preliminary user study (N =20) indicate that effectiveness and probability of item selection increase when users can explore relations between multiple recommendations and human feedback. In addition, perceived effectiveness and actual effectiveness of the recommendations as well as user trust into the recommendations are higher than a traditional list representation of recommendations

Supporting conference attendees with visual decision making interfaces

Recent efforts in recommender systems research focus increasingly on human factors affecting recommendation acceptance, such as transparency and user control. In this paper, we present IntersectionExplorer, a scalable visualization to interleave the output of several recommender engines with user-contributed relevance information, such as bookmarks and tags. Two user studies at conferences indicate that this approach is well suited for technical audiences in smaller venues, and allowed the identification of applicability limitations for less technical audiences attending larger events. Copyright held by the owner/author(s)

A Survey on Linked Data and the Social Web as facilitators for TEL recommender systems

Personalisation, adaptation and recommendation are central features of TEL environments. In this context, information retrieval techniques are applied as part of TEL recommender systems to filter and recommend learning resources or peer learners according to user preferences and requirements. However, the suitability and scope of possible recommendations is fundamentally dependent on the quality and quantity of available data, for instance, metadata about TEL resources as well as users. On the other hand, throughout the last years, the Linked Data (LD) movement has succeeded to provide a vast body of well-interlinked and publicly accessible Web data. This in particular includes Linked Data of explicit or implicit educational nature. The potential of LD to facilitate TEL recommender systems research and practice is discussed in this paper. In particular, an overview of most relevant LD sources and techniques is provided, together with a discussion of their potential for the TEL domain in general and TEL recommender systems in particular. Results from highly related European projects are presented and discussed together with an analysis of prevailing challenges and preliminary solutions.LinkedU

Assessing the validity of a learning analytics expectation instrument: A multinational study

To assist Higher Education Institutions in meeting the challenge of limited student engagement in the implementation of Learning Analytics services, the Questionnaire for Student Expectations of Learning Analytics (QSELA) was developed. This instrument contains 12 items, which are explained by a purported two-factor structure of Ethical and Privacy Expectations and Service Expectations. As it stands, however, the QSELA has only been validated with students from UK University students, which is problematic on account of the interest in Learning Analytics extending beyond this context. Thus, the aim of the current work was to assess whether the translated QSELA can be validated in three contexts (an Estonian, a Spanish, and a Dutch University). The findings show that the model provided acceptable fits in both the Spanish and Dutch samples, but was not supported in the Estonian student sample. In addition, an assessment of local fit is undertaken for each sample, which provides important points that need to be considered in future work. Finally, a general comparison of expectations across contexts is undertaken, which are discussed in relation to the General Data Protection Regulation (GDPR, 2018)

Challenges in context-aware mobile language learning: the MASELTOV approach

Smartphones, as highly portable networked computing devices with embedded sensors including GPS receivers, are ideal platforms to support context-aware language learning. They can enable learning when the user is en-gaged in everyday activities while out and about, complementing formal language classes. A significant challenge, however, has been the practical implementation of services that can accurately identify and make use of context, particularly location, to offer meaningful language learning recommendations to users. In this paper we review a range of approaches to identifying context to support mobile language learning. We consider how dynamically changing aspects of context may influence the quality of recommendations presented to a user. We introduce the MASELTOV project’s use of context awareness combined with a rules-based recommendation engine to present suitable learning content to recent immigrants in urban areas; a group that may benefit from contextual support and can use the city as a learning environment